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DETAILED ACTION 
Specification 

Applicant is reminded of the proper language and format for an abstract of 
the disclosure. 

The abstract should be in narrative form and generally limited to a single 
paragraph on a separate sheet within the range of 50 to 150 words. It is 
important that the abstract not exceed 150 words in length since the space 
provided for the abstract on the computer tape used by the printer is limited. The 
form and legal phraseology often used in patent claims, such as "means" and 
"said," should be avoided. The abstract should describe the disclosure 
sufficiently to assist readers in deciding whether there is a need for consulting the 
full patent text for details. 

The language should be clear and concise and should not repeat 
information given in the title. It should avoid using phrases which can be implied, 
such as, "The disclosure concerns," "The disclosure defined by this invention," 
"The disclosure describes," etc. 

More detail should be provided within the abstract so as to describe the 
disclosure sufficiently to assist readers in deciding whether there is a need for 
consulting the full patent text for details. 



Drawings 

The drawings are objected to under 37 CFR 1 .83(a) because they fail to 
show recognizer 21 1 as described in the specification, page 14, line 25. Any 
structural detail that is essential for a proper understanding of the disclosed 
invention should be shown in the drawing. MPEP § 608.02(d). Corrected drawing 
sheets in compliance with 37 CFR 1.121(d) are required in reply to the Office 
action to avoid abandonment of the application. Any amended replacement 
drawing sheet should include all of the figures appearing on the immediate prior 
version of the sheet, even if only one figure is being amended. The figure or 
figure number of an amended drawing should not be labeled as "amended." If a 
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drawing figure is to be canceled, the appropriate figure must be removed from 
the replacement sheet, and where necessary, the remaining figures must be 
renumbered and appropriate changes made to the brief description of the several 
views of the drawings for consistency. Additional replacement sheets may be 
necessary to show the renumbering of the remaining figures. The replacement 
sheet(s) should be labeled "Replacement Sheet" in the page header (as per 37 
CFR 1 .84(c)) so as not to obstruct any portion of the drawing figures. If the 
changes are not accepted by the examiner, the applicant will be notified and 
informed of any required corrective action in the next Office action. The objection 
to the drawings will not be held in abeyance. 

Claim Objections 

Claim 25 is objected to because of the following informalities: 
♦ The word "for" is extraneous and should be removed. 
Appropriate correction is required. 

Claim Rejections - 35 USC § 102 

The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(e) the invention was described in (1) an application for patent, published under section 
122(b), by another filed in the United States before the invention by the applicant for patent or 
(2) a patent granted on an application for patent by another filed in the United States before 
the invention by the applicant for patent, except that an international application filed under 
the treaty defined in section 351(a) shall have the effects for purposes of this subsection of an 
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application filed in the United States only if the international application designated the United 
States and was published under Article 21(2) of such treaty in the English language. 

Claims 1,3-6, 8-9, 26, 28-30, 32, 35, 37, 40-43, 45-50, and 56-57 are 
rejected under 35 U.S.C. 102(e) as being anticipated by Barclay (U.S. Patent 
5,960,399). 

As per claim 1 , Barclay discloses a computer readable medium having a 
markup language for execution on a client device in a client/server system where 
the markup language includes an instruction indicating a grammar to associate 
with input entered through the client device. (See Barclay, Column 8, lines 26- 
28). 

As per claim 3, Barclay discloses that the grammar is for speech 
recognition, which is included in the list of possible grammars set forth in claim 3. 
(See Barclay, Column 8, lines 26-28). 

As per claim 4, Barclay discloses that the instruction indicating a grammar 
includes a reference to a grammar for speech recognition. (See Barclay, Column 
8, lines 26-28). 

As per claim 5, Barclay discloses that the markup language is HTML, 
which is included in the list of possible markup languages set forth in claim 5. 
(See Barclay, Column 8, lines 42-46). 

As per claim 6, Barclay discloses that the markup language includes a 
scripting language, in the form of an embedded Java applet. (See Barclay, 
Column 8, lines 48-51). 

As per claim 8, Barclay discloses that the markup language includes an 
instruction for receiving a recognition result pertaining to recognized speech and 
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associating the result with a data field on the client device. (See Barclay, 
Column 8, lines 48-64). 

As per claim 9, Barclay discloses that the instruction for receiving a 
recognition result associates the recognition result to a number of data fields. 
(See Barclay, Column 8, lines 65-67, and Column 9, lines 1-15). 

As per claim 26, Barclay discloses a computer readable medium including 
instructions readable by a computer which, when implemented, cause the 
computer to handle information by performing steps including receiving a markup 
language page from a web server having a field for input data (See Barclay, 
Column 8, lines 65-67, and Column 9, lines 1-15), receiving input from a user 
related to the field (See Barclay, Column 9, lines 9-15), and sending data 
indicative of the input and an indication of a grammar for recognition. (See 
Barclay, Column 9, lines 16-20). 

As per claim 28, Barclay discloses that the indication includes a reference 
to a language for speech recognition. (See Barclay, Column 9, lines 34-38). 

Claim 29 is rejected on the same basis as claim 5. 

Claim 30 is rejected on the same basis as claim 6. 

As per claim 32, Barclay discloses that the input includes speech, and 
also includes instructions which, when implemented, include processing the input 
speech to provide data indicative of the input speech. (See Barclay, Column 9, 
lines 16-30). 
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As per claim 35, Barclay discloses that the input includes speech and the 
grammar includes a speech recognition grammar. (See Barclay, Column 8, lines 
22-28). 

Claim 37 is rejected on the same basis as claim 26. 

Claim 40 is rejected on the same basis as claim 5. 

Claim 41 is rejected on the same basis as claim 8. 

Claim 42 is rejected on the same basis as claim 9. 

Claim 43 is rejected on the same basis as claim 6. 

As per claim 45, Barclay discloses a computer readable medium having a 
markup language for execution on a client device in a client/server system, 
where the markup language includes an instruction indicating an object model 
element having attributes for recognition. (See Barclay, Column 4, lines 57-67, 
Column 5, lines 1-11, and Column 8, lines 26-28 and lines 42-48). 

Claim 46 is rejected on the same basis as claim 3. 

Claim 47 is rejected on the same basis as claim 4. 

Claim 48 is rejected on the same basis as claim 8. 

Claim 49 is rejected on the same basis as claim 9. 

As per claim 50, Barclay discloses that the markup language includes 
instructions for handling an event, which is an action or occurrence detected by a 
program (www.webopedia.com). (See Barclay, Column 8, lines 48-64). 

Claim 56 is rejected on the same basis as claim 5. 

Claim 57 is rejected on the same basis as claim 6. 
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Claim Rejections - 35 USC § 103 

The following is a quotation of 35 U.S.C. 103(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 102 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

Claims 2 and 39 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Barclay (U.S. Patent 5,960,399) as applied to claims 1 and 37 
above, and further in view of Wohlsen (U.S. Publication 2003/0125944 A1). 

As per claims 2 and 39, Barclay discloses the limitations of claims 1 and 
37 as described above. Barclay does not disclose expressly that the instruction 
indicating a grammar provides a reference to a location of the grammar. 
Wohlsen discloses that the grammar is stored in a special location in grammar 
storage. (See Wohlsen, Page 3, paragraph 0037). Barclay and Wohlsen are 
analogous art because they are from the same field of endeavor of developing 
speech recognition systems. At the time of the invention it would have been 
obvious to a person of ordinary skill in the art to include the location for the 
grammar of Wohlsen with the grammar of Barclay. The motivation for doing so 
would have been to provide a location from which stored grammars can be 
matched to inputted data for comparison. (See Wohlsen, Page 1 , paragraph 
0007). Therefore, it would have been obvious to combine Wohlsen with Barclay 
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for the benefit of providing a storage location for previously inputted grammars to 
obtain the invention as specified in claims 2 and 39. 

Claims 7, 27, 31, 44, and 58 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Barclay (U.S. Patent 5,960,399) as applied to claims 4, 26, 37, 
and 45 above, and further in view of Holtz (U.S. Publication 2002/0031 756 A1 ). 

As per claims 7, 27, 31, 44 and 58, Barclay discloses the limitations of 
claims 4, 26, 37 and 45 as described above. Barclay does not disclose expressly 
that the markup language includes synchronized multimedia markup language. 
Holtz discloses that the system can be configured to support Synchronized 
Multimedia Integration Markup Language (SMIL). (See Holtz, Page 9-10, 
paragraph 0129). Barclay and Holtz are analogous art because they are from 
the same field of endeavor of processing data over the web using a markup 
language. At the time of the invention it would have been obvious to a person of 
ordinary skill in the art to include the Synchronized Multimedia Integration 
Markup Language (SMIL) of Holtz with the instruction indicating a grammar for 
speech recognition of Barclay. The motivation for doing so would have been to 
make the media streams compatible with the client workstations. (See Holtz, 
Page 10, paragraph 0129). Therefore, it would have been obvious to combine 
Holtz with Barclay for the benefit of client workstation compatibility to obtain the 
invention as specified in claims 7, 27, 31 , 44 and 58. 
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Claims 10-11 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Barclay (U.S. Patent 5,960,399) as applied to claim 1 above, and further in 
view of Baruch (U.S. Publication 2002/0091 51 8 A1 ). 

As per claim 10, Barclay discloses the limitations of claim 1 as described 
above. Barclay does not disclose expressly that the markup language includes 
an instruction to audibly render an indication when the speech was not 
recognized. Baruch discloses that if the voice input is not recognized, the system 
may provide an audible (via loudspeaker) message to the user. (See Baruch, 
Page 4, paragraph 0041). Barclay and Baruch are analogous art because they 
are from the same field of endeavor of utilizing speech recognition systems for 
user input and output. At the time of the invention it would have been obvious to 
a person of ordinary skill in the art to include the audible response of Baruch with 
the markup language instructions of Barclay. The motivation for doing so would 
have been to provide audio feedback to the user. (See Baruch, Page 1 , 
paragraph 0007). Therefore, it would have been obvious to combine Baruch with 
Barclay for the benefit of providing an audio response to the user to obtain the 
invention as specified in claim 10. 

As per claim 1 1 , Barclay discloses the limitations of claim 1 as described 
above. Barclay does not disclose expressly that the markup language includes 
an instruction to visually render an indication when the speech was not 
recognized. Baruch discloses that if the voice input is not recognized, the system 
may provide a visual (via display unit) message to the user. (See Baruch, Page 
4, paragraph 0041 ). Barclay and Baruch are analogous art because they are 
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from the same field of endeavor of utilizing speech recognition systems for user 
input and output. At the time of the invention it would have been obvious to a 
person of ordinary skill in the art to include the visual response of Baruch with the 
markup language instructions of Barclay. The motivation for doing so would 
have been to provide visual feedback to the user. (See Baruch, Page 1 , 
paragraph 0007). Therefore, it would have been obvious to combine Baruch with 
Barclay for the benefit of providing a visual response to the user to obtain the 
invention as specified in claim 1 1 . 

Claims 18-20, 25, 36, and 52 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Barclay (U.S. Patent 5,960,399) as applied to claims 1 
and 50 above, and further in view of Ladd (U.S. Patent 6,385,583 B1 ). 

As per claim 18, Barclay discloses the limitations of claim 1 as described 
above. Barclay does not disclose expressly that the grammar includes a DTMF 
grammar. Ladd discloses that inputted DTMF tones are included in the 
processing of the speech communications. (See Ladd, Column 7, lines 61-67, 
Column 8, lines 1-3, and Column 14, lines 6-14). Barclay and Ladd are 
analogous art because they are from the same field of endeavor of developing 
markup languages for services in which clients interact with a server. At the time 
of the invention it would have been obvious to a person of ordinary skill in the art 
to include the DTMF grammar of Ladd with the grammar of Barclay. The 
motivation for doing so would have been to allow input from the user via DTMF 
tones. (See Ladd, Column 14, lines 7-9). Therefore, it would have been obvious 
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to combine Ladd with Barclay for the benefit of allowing DTMF tone input to 
obtain the invention as specified in claim 18. 

As per claim 19, Barclay discloses the limitations of claim 1 as described 
above. Barclay does not disclose expressly that the markup language includes 
an instruction for handling a DTMF event. Ladd discloses that the system 
includes processing of events from the user inputted via speech or DTMF tones. 
(See Ladd, Column 4, lines 28-40). Barclay and Ladd are analogous art because 
they are from the same field of endeavor of developing markup languages for 
services in which clients interact with a server. At the time of the invention it 
would have been obvious to a person of ordinary skill in the art to include the 
processing of DTMF tones of Ladd with the markup language instructions of 
Barclay. The motivation for doing so would have been to allow input from the 
user via DTMF tones. (See Ladd, Column 14, lines 7-9). Therefore, it would 
have been obvious to combine Ladd with Barclay for the benefit of allowing 
DTMF tone input to obtain the invention as specified in claim 18. 

As per claim 20, Barclay discloses the limitations of claim 1 as described 
above. Barclay does not disclose expressly that the markup language includes 
an instruction for handling a call control event. Ladd discloses a system that 
handles call control events via a call control unit. (See Ladd, Column 8, lines 26- 
36). Barclay and Ladd are analogous art because they are from the same field of 
endeavor of developing markup languages for services in which clients interact 
with a server. At the time of the invention it would have been obvious to a person 
of ordinary skill in the art to include the call control instructions of Ladd with the 
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markup language of Barclay. The motivation for doing so would have been to 
allow users to access the system from any number of locations using a variety of 
input devices. (See Ladd, Column 2, lines 19-39). Therefore, it would have been 
obvious to combine Ladd with Barclay for the benefit of allowing access to the 
system via a large number of locations and input devices to obtain the invention 
as specified in claim 20. 

As per claim 25, Barclay and Ladd disclose the limitations of claim 20 as 
described above. Ladd also discloses that the markup language includes an 
instruction related to halting prompting using a "barge-in" attribute. (See Ladd, 
Column 17, lines 31-41). ). Barclay and Ladd are analogous art because they 
are from the same field of endeavor of developing markup languages for services 
in which clients interact with a server. At the time of the invention it would have 
been obvious to a person of ordinary skill in the art to include the "barge-in" 
attribute of Ladd with the markup language of Barclay and Ladd. The motivation 
for doing so would have been to allow the processing to be interrupted at any 
time based on a predetermined response from the user. (See Ladd, Column 17, 
lines 39-41 ). Therefore, it would have been obvious to combine Ladd with 
Barclay and Ladd for the benefit of interrupting processing to obtain the invention 
as specified in claim 25. 

Claim 36 is rejected on the same basis as claim 18. 

As per claim 52, Barclay discloses the limitations of claim 50 as described 
above. Barclay does not disclose expressly that the event pertains to whether 
recognition was obtained. Ladd discloses an event, such as the displaying of an 
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error message, which is activated if the voice browser has difficulty recognizing 
inputs from the user. (See Ladd, Column 14, lines 40-53). Barclay and Ladd 
are analogous art because they are from the same field of endeavor of 
developing markup languages for services in which clients interact with a server. 
At the time of the invention it would have been obvious to a person of ordinary 
skill in the art to include the event pertaining to whether recognition was obtained 
of Ladd with the markup language instructions of Barclay. The motivation for 
doing so would have been to activate error handling code used to prompt the 
user to repeat his or her input. (See Ladd, Column 14, lines 54-60). Therefore, it 
would have been obvious to combine Ladd with Barclay for the benefit of 
enabling error handling in the event the input is not recognized to obtain the 
invention as specified in claim 52. 

Claims 12-16, 33-34, 38, and 51 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over Barclay (U.S. Patent 5,960,399) as applied to claims 1 , 
32, 37, and 50 above, and further in view of Andreshak (U.S. Patent 5,664,061). 

As per claims 12 and 51 , Barclay discloses the limitations of claims 1 and 
45 as described above. Barclay does not disclose expressly that the markup 
language includes an instruction to set a time period related to speech 
recognition. Andreshak discloses that a speech recognizer measures the value 
of at least one feature of an utterance during each of a series of successive time 
intervals within a second time period. (See Andreshak, Column 3, lines 31-35). 
Barclay and Andreshak are analogous art because they are from the same field 



Application/Control Number: 09/960,233 Page 
Art Unit: 2176 

of endeavor of utilizing speech recognition systems for user input and output. At 
the time of the invention it would have been obvious to a person of ordinary skill 
in the art to include the time period of Andreshak with the markup language 
instructions of Barclay. The motivation for doing so would have been to allow 
time for a sufficient amount of input data to be entered into the system in order to 
determine a match between the input and the stored vocabulary or grammar. 
(See Andreshak, Column 3, lines 34-41). Therefore, it would have been obvious 
to combine Andreshak with Barclay for the benefit of processing the input data to 
obtain the invention as specified in claims 12 and 51. 

As per claim 13, Barclay and Andreshak disclose the limitations of claim 
12 as described above. Andreshak also discloses that the time period indicates 
a maximum period of silence from a start recognition instruction. (See 
Andreshak Column 13, lines 1-10). Barclay and Andreshak are analogous art 
because they are from the same field of endeavor of utilizing speech recognition 
systems for user input and output. At the time of the invention it would have 
been obvious to a person of ordinary skill in the art to include the maximum time 
period of silence of Andreshak with the time period of Barclay and Andreshak. 
The motivation for doing so would have been to allow time for distinguishing 
silence from background noise. (See Andreshak, Column 12, lines 58-66). 
Therefore, it would have been obvious to combine Andreshak with Barclay for the 
benefit of determining whether the input is noise or silence to obtain the invention 
as specified in claim 13. 
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As per claim 14, Barclay and Andreshak disclose the limitations of claim 
12 as described above. Barclay also discloses that the time period indicates a 
maximum period before a result must be returned after detection of speech. 
(See Barclay, Column 7, lines 9-26). 

As per claim 15, Barclay and Andreshak disclose the limitations of claim 
12 as described above. Barclay also discloses that the time period indicates a 
maximum period before a result must be returned after a start recognition 
instruction. (See Barclay, Column 7, lines 9-26). 

As per claim 16, Barclay and Andreshak disclose the limitations of claim 
12 as described above. Barclay also discloses that the time period indicates a 
minimum period of silence to indicate an end of speech. (See Barclay, Column 
7, lines 27-40). 

As per claim 33, Barclay discloses the limitations of claim 32 as described 
above. Barclay does not disclose expressly that the processing includes 
normalizing the data indicative of the input speech. Andreshak discloses a 
normalization processor to normalize the input speech. (See Andreshak, Column 
13, lines 23-35). Barclay and Andreshak are analogous art because they are 
from the same field of endeavor of utilizing speech recognition systems for user 
input and output. At the time of the invention it would have been obvious to a 
person of ordinary skill in the art to include the normalization of the input speech 
of Andreshak with the processing instructions of Barclay. The motivation for 
doing so would have been to adjust for variations in the loudness of the input 
speech. (See Andreshak, Column 13, lines 20-22). Therefore, it would have 
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been obvious to combine Andreshak with Barclay for the benefit of adjusting the 
volume level of the input speech to obtain the invention as specified in claim 33. 

As per claim 34, Barclay discloses the limitations of claim 32 as described 
above. Barclay does not disclose expressly that the processing includes 
generating data indicative of parameters of the input speech. Andreshak 
discloses generating arguments in the form of parameters input to the function 
which are indicative of the input speech. (See Andreshak, Columns 27-29, 
example IV). Barclay and Andreshak are analogous art because they are from 
the same field of endeavor of utilizing speech recognition systems for user input 
and output. At the time of the invention it would have been obvious to a person 
of ordinary skill in the art to include the parameters of Andreshak with the 
processing instructions of Barclay. The motivation for doing so would have been 
to define the vocabulary or grammar. (See Andreshak, Column 27, lines 24-26). 
Therefore, it would have been obvious to combine Andreshak with Barclay for the 
benefit of defining the vocabulary or grammar to obtain the invention as specified 
in claim 34. 

As per claim 38, Barclay discloses the limitations of claim 37 as described 
above. Barclay does not disclose expressly normalizing the data of the input 
prior to sending the data. Andreshak discloses a normalization processor to 
normalize the input speech. (See Andreshak, Column 13, lines 23-35). Barclay 
and Andreshak are analogous art because they are from the same field of 
endeavor of utilizing speech recognition systems for user input and output. At 
the time of the invention it would have been obvious to a person of ordinary skill 
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in the art to include the normalization of the input speech of Andreshak with the 
processing instructions of Barclay. The motivation for doing so would have been 
to adjust for variations in the loudness of the input speech. (See Andreshak, 
Column 13, lines 20-22). Therefore, it would have been obvious to combine 
Andreshak with Barclay for the benefit of adjusting the volume level of the input 
speech to obtain the invention as specified in claim 38. 

Claim 17 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Barclay (U.S. Patent 5,960,399) as applied to claim 1 above, and further in view 
of Brems (U.S. Patent 5,566,272). 

As per claim 17, Barclay discloses the limitations of claim 1 as described 
above. Barclay does not disclose expressly that the markup language includes 
an instruction indicating a confidence measure to use for speech recognition. 
Brems discloses the use of a confidence measure for speech recognition 
processing. (See Brems, Figures 2 and 3) Barclay and Brems are analogous art 
because they are from the same field of endeavor of developing speech 
recognition systems. At the time of the invention it would have been obvious to a 
person of ordinary skill in the art to include the confidence measure of Brems 
with the processing of Barclay. The motivation for doing so would have been to 
better handle recognition errors. (See Brems, Column 1, lines 25-52). Therefore, 
it would have been obvious to combine Brems with Barclay for the benefit of 
reducing input speech recognition errors to obtain the invention as specified in 
claim 17. 
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Claims 21, 55, and 59-61 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Barclay (U.S. Patent 5,960,399) as applied to claims 1 and 45 
above, and further in view of Sturner (U.S. Patent 5,303,327). 

As per claims 21 and 55, Barclay discloses the limitations of claims 1 and 
45 as described above. Barclay does not disclose expressly that the markup 
language includes an instruction indicating for audibly prompting. Sturner 
discloses an audio prompt to ask to user to repeat his or her input. (See 
Sturner, Column 9, lines 46-49). Barclay and Sturner are analogous art because 
they are from the same field of endeavor of utilizing speech recognition systems 
for user input and output. At the time of the invention it would have been obvious 
to a person of ordinary skill in the art to include the audio prompt of Sturner with 
the markup language of Barclay. The motivation for doing so would have been to 
prompt the user to repeat the verbal input. (See Sturner, Column 9, lines 48-49). 
Therefore, it would have been obvious to combine Sturner with Barclay for the 
benefit of prompting the user to repeat his or her input to obtain the invention as 
specified in claims 21 and 55. 

As per claim 59, Barclay discloses a computer readable medium having a 
markup language for execution on a client device in a client/server system that 
includes an instruction set indicating an object model element. (See Barclay, 
Column 4, lines 57-67, Column 5, lines 1-11, and Column 8, lines 26-28 and lines 
42-48). Barclay does not disclose expressly the object model element including 
attributes for audibly prompting. Sturner discloses attributes for audibly 
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prompting. (See Sturner, Column 9, lines 46-49). Barclay and Stumer are 
analogous art because they are from the same field of endeavor of utilizing 
speech recognition systems for user input and output. At the time of the 
invention it would have been obvious to a person of ordinary skill in the art to 
include the audio prompt of Sturner with the markup language of Barclay. The 
motivation for doing so would have been to prompt the user to repeat the verbal 
input. (See Sturner, Column 9, lines 48-49). Therefore, it would have been 
obvious to combine Sturner with Barclay for the benefit of prompting the user to 
repeat his or her input to obtain the invention as specified in claim 59. 

Claim 60 is rejected on the same basis as claim 5. 

Claim 61 is rejected on the same basis as claim 6 

Claims 22-24 and 63 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Barclay (U.S. Patent 5,960,399) and Sturner (U.S. Patent 
5,303,327) as applied to claims 21 and 59 above, and further in view of Ball (U.S. 
Patent 6,240,391 B1). 

As per claim 22, Barclay and Sturner disclose the limitations of claim 21 
as described above. Barclay and Sturner do not disclose expressly that the 
markup language includes an instruction for prompting by converting text to 
speech. Ball discloses converting text to speech using a text-to-speech 
processor. (See Ball, Column 3, lines 34-36). Barclay, Sturner and Ball are 
analogous art because they are from the same field of endeavor of utilizing 
speech recognition systems for user input and output. At the time of the 
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invention it would have been obvious to a person of ordinary skill in the art to 
include the text-to-speech processing of Ball with the markup language of 
Barclay and Sturner. The motivation for doing so would have been to allow the 
textual fragments to be retrieved from a designated address. (See Ball, Column 
3, lines 9-14). Therefore, it would have been obvious to combine Ball with 
Barclay and Sturner for the benefit of improved data storage to obtain the 
invention as specified in claim 22. 

As per claim 23, Barclay and Sturner disclose the limitations of claim 21 
as described above. Barclay and Sturner do not disclose expressly that the 
markup language includes an instruction for prompting by playing a selected 
audio stream. Ball discloses playing an audio file. (See Ball, Column 3, lines 36- 
39). Barclay, Sturner and Ball are analogous art because they are from the 
same field of endeavor of utilizing speech recognition systems for user input and 
output. At the time of the invention it would have been obvious to a person of 
ordinary skill in the art to include the audio file of Ball with the markup language 
and prompt of Barclay and Sturner. The motivation for doing so would have 
been to provide a verbal prompt to the user. (See Ball, Column 3, lines 36-39). 
Therefore, it would have been obvious to combine Ball with Barclay and Sturner 
for the benefit of providing a verbal prompt to the user to obtain the invention as 
specified in claim 23. 

As per claim 24, Barclay and Sturner disclose the limitations of claim 21 
as described above. Barclay and Sturner do not disclose expressly that the 
markup language includes an instruction for retrieving a value for use in 
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prompting. Ball discloses retrieving a user's touch-tone keypad inputs. (See 
Ball, Column 3, lines 39-47). Barclay, Sturner and Ball are analogous art 
because they are from the same field of endeavor of utilizing speech recognition 
systems for user input and output. At the time of the invention it would have 
been obvious to a person of ordinary skill in the art to include the touch-tone 
keypad inputs of Ball with the markup language instructions of Barclay and 
Sturner. The motivation for doing so would have been to effect interaction and 
navigation within the structured message as allowed by the markup within the 
message. (See Ball, Column 3, lines 44-47). Therefore, it would have been 
obvious to combine Ball with Barclay and Sturner for the benefit of allowing 
interaction and navigation within the message to obtain the invention as specified 
in claim 24. 

As per claim 63, Barclay and Sturner disclose the limitations of claim 59 
as described above. Barclay and Sturner do not disclose expressly that the 
element includes playing an audio file. Ball discloses playing an audio file. (See 
Ball, Column 3, lines 36-39). Barclay, Sturner and Ball are analogous art 
because they are from the same field of endeavor of utilizing speech recognition 
systems for user input and output. At the time of the invention it would have 
been obvious to a person of ordinary skill in the art to include the audio file of Ball 
with the markup language and prompt of Barclay and Sturner. The motivation for 
doing so would have been to provide a pre-recorded response to the user. (See 
Ball, Column 8, lines 28-31). Therefore, it would have been obvious to combine 
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Ball with Barclay and Sturner for the benefit of providing a pre-recorded response 
to the user to obtain the invention as specified in claim 63. 

Claims 66-68 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over Barclay (U.S. Patent 5,960,399) and Sturner (U.S. Patent 5,303,327) as 
applied to claims 21 and 59 above, and further in view of Ladd (U.S. Patent 
6,385,583 B1). 

As per claim 66, Barclay and Sturner disclose the limitations of claim 59 
as described above. Barclay and Sturner do not disclose expressly that one of 
the attributes includes a period of time when prompting can be interrupted. Ladd 
discloses that the markup language includes an instruction related to halting 
prompting at any time using a "bargein" attribute. (See Ladd, Column 17, lines 
31-41). ). Barclay, Sturner, and Ladd are analogous art because they are from 
the same field of endeavor of developing markup languages for services in which 
clients interact with a server. At the time of the invention it would have been 
obvious to a person of ordinary skill in the art to include the bargein attribute of 
Ladd with the markup language of Barclay and Sturner. The motivation for doing 
so would have been to allow the processing to be interrupted at any time based 
on a predetermined response from the user. (See Ladd, Column 17, lines 39- 
41). Therefore, it would have been obvious to combine Ladd with Barclay and 
Sturner for the benefit of interrupting processing to obtain the invention as 
specified in claim 66. 
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As per claim 67, Barclay and Sturner disclose the limitations of claim 59 
as described above. Barclay and Sturner do not disclose expressly that the 
markup language includes instructions for handling an event related to 
prompting. Ladd discloses that the markup language includes a PROMPT 
element. (See Ladd, Column 17, lines 65-67, and Column 18, lines 1-21). 
Barclay, Sturner, and Ladd are analogous art because they are from the same 
field of endeavor of developing markup languages for services in which clients 
interact with a server. At the time of the invention it would have been obvious to 
a person of ordinary skill in the art to include the PROMPT element of Ladd with 
the markup language of Barclay and Sturner. The motivation for doing so would 
have been to define the application state. (See Ladd, Column 18, lines 9-10). 
Therefore, it would have been obvious to combine Ladd with Barclay and Sturner 
for the benefit of defining the state of the application to obtain the invention as 
specified in claim 67. 

As per claim 68, Barclay, Sturner, and Ladd disclose the limitations of 
claim 67 as described above. Ladd also discloses an event that pertains to 
whether prompting is being interrupted in the form of a "BARGEIN" element. 
(See Ladd, Column 17, lines 65-67, and Column 18, lines 1-8). Barclay, Sturner, 
and Ladd are analogous art because they are from the same field of endeavor of 
developing markup languages for services in which clients interact with a server. 
At the time of the invention it would have been obvious to a person of ordinary 
skill in the art to include the "BARGEIN" element of Ladd with the markup 
language of Barclay, Sturner and Ladd. The motivation for doing so would have 
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been to allow the processing to be interrupted at any time based on a 
predetermined response from the user. (See Ladd, Column 17, lines 39-41). 
Therefore, it would have been obvious to combine Ladd with Barclay, Sturner, 
and Ladd for the benefit of interrupting processing to obtain the invention as 
specified in claim 68. 

Claim 62 is rejected under 35 U.S.C. 103(a) as being unpatentable over 
Barclay (U.S. Patent 5,960,399) and Sturner (U.S. Patent 5,303,327) as applied 
to claim 59 above, and further in view of Holtz (U.S. Publication 2002/0031756 
A1). 

As per claim 62, Barclay and Sturner disclose the limitations of claim 59 
as described above. Barclay and Sturner do not disclose expressly that the 
markup language includes a synchronized markup language. Holtz discloses 
that the system can be configured to support Synchronized Multimedia 
Integration Markup Language (SMIL). (See Holtz, Page 9-10, paragraph 0129). 
Barclay, Sturner, and Holtz are analogous art because they are from the same 
field of endeavor of processing data over the web using a markup language. At 
the time of the invention it would have been obvious to a person of ordinary skill 
in the art to include the Synchronized Multimedia Integration Markup Language 
(SMIL) of Holtz with the markup language of Barclay and Sturner. The 
motivation for doing so would have been to make the media streams compatible 
with the client workstations. (See Holtz, Page 10, paragraph 0129). Therefore, it 
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would have been obvious to combine Holtz with Barclay for the benefit of client 
workstation compatibility to obtain the invention as specified in claim 62. 

Claims 53 and 54 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Barclay (U.S. Patent 5,960,399) as applied to claim 50 above, 
and further in view of Fox (U.S. Publication 2002/0077823 A1 ). 

As per claim 53, Barclay discloses the limitations of claim 50 as described 
above. Barclay does not disclose expressly an event pertaining to when a DTMF 
key is pressed. Fox discloses simulating voice interactions with an end user 
through the use of DTMF buttons. (See Fox, Page 7, paragraph 0077). Barclay 
and Fox are analogous art because they are from the same field of endeavor of 
utilizing speech recognition systems for user input and output. At the time of the 
invention it would have been obvious to a person of ordinary skill in the art to 
include the DTMF input of Fox with the event processing of Barclay. The 
motivation for doing so would have been to allow the developer to enter text that 
would correspond to what an end user would say in response to a prompt from 
the application. (See Fox, Page 7, paragraph 0077) Therefore, it would have 
been obvious to combine Fox with Barclay for the benefit of inputting data via 
DTMF keys rather than speech to obtain the invention as specified in claim 53. 

As per claim 54, Barclay and Fox disclose the limitations of claim 53 as 
described above. Fox also discloses that the event pertains to call control. (See 
Fox, Page 7, paragraph 0077) Barclay and Fox are analogous art because they 
are from the same field of endeavor of utilizing speech recognition systems for 
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user input and output. At the time of the invention it would have been obvious to 
a person of ordinary skill in the art to include the call control events of Fox with 
the event of Barclay. The motivation for doing so would have been to allow a 
developer to simulate a call initiation. (See Fox, Page 7, paragraph 0077) 
Therefore, it would have been obvious to combine Fox with Barclay for the 
benefit of simulating call initiation to obtain the invention as specified in claim 54. 

Claims 64 and 65 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Barclay (U.S. Patent 5,960,399) and Stumer (U.S. Patent 
5,303,327) as applied to claim 59 above, and further in view of Fox (U.S. 
Publication 2002/0077823 A1). 

As per claim 64, Barclay and Sturner disclose the limitations of claim 59 
as described above. Barclay and Sturner do not disclose expressly that the 
element includes converting text-to-speech. Fox discloses converting text-to- 
speech. (See Fox, Page 6, paragraph 0071) Barclay, Sturner and Fox are 
analogous art because they are from the same field of endeavor of utilizing 
speech recognition systems for user input and output. At the time of the 
invention it would have been obvious to a person of ordinary skill in the art to 
include the conversion of text-to-speech of Fox with the markup language 
instructions of Barclay and Sturner. The motivation for doing so would have 
been to aurally describe the contents of a web page to an end user. (See Fox, 
Page 6, paragraph 0071 ) Therefore, it would have been obvious to combine Fox 
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with Barclay and Sturner for the benefit of aurally communicating data to the end 
user to obtain the invention as specified in claim 64. 

As per claim 65, Barclay, Sturner and Fox disclose the limitations of claim 
64 as described above. Fox also discloses that the element obtains dynamic 
content for converting text-to-speech. (See Fox, Page 6, paragraph 0071 , and 
Page 7, paragraph 0075) Barclay, Sturner and Fox are analogous art because 
they are from the same field of endeavor of utilizing speech recognition systems 
for user input and output. At the time of the invention it would have been obvious 
to a person of ordinary skill in the art to include the conversion of dynamic 
content from text to speech with the markup language instructions of Barclay and 
Sturner. The motivation for doing so would have been to aurally describe the 
contents of a web page to an end user. (See Fox, Page 6, paragraph 0071 ) 
Therefore, it would have been obvious to combine Fox with Barclay and Sturner 
for the benefit of aurally communicating data to the end user to obtain the 
invention as specified in claim 65. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. 

• Albayrak (U.S. Patent 6,662,163 B1) discloses a system and 
method for programming portable devices from a remote computer 
system. 
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• Brown (U.S. Patent 6,604,075 B1 ) discloses a web-based voice 
dialog interface. 

• Baker (U.S. Patent 6,456,974 B1 ) discloses a system and method 
for adding speech recognition capabilities to Java. 

• Giangarra (U.S. Patent 6,101,472) discloses a data processing 
system and method for navigating a network using voice command. 

• Dragosh (U.S. Patent 6,078,886) discloses a system and method 
for providing remote automatic speech recognition services via a 
packet network. 

• Chou (U.S. Patent 5,737,489) discloses a system and method of 
discriminative utterance verification for connected digits recognition. 

Any inquiry concerning this communication or earlier communications from 
the examiner should be directed to Laurie Ries whose telephone number 
is currently (703) 605-1238. After mid-October, 2004, the examiner can 
be reached at (571 ) 272-4095. The examiner can normally be reached on 
Monday-Friday from 7:00am to 3:30pm. 

Information regarding the status of an application may be obtained from 
the Patent Application Information Retrieval (PAIR) system. Status 
information for published applications may be obtained from either Private 
PAIR or Public PAIR. Status information for unpublished applications is 
available through Private PAIR only. For more information about the 
PAIR system, see http://pair-direct.uspto.gov . Should you have any 
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questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). 
LAR 




